lorrie faith cranor: what's wrong with your pa$$w0rd?

I am a computer science and engineering professor here at Carnegie Mellon, and my research focuses on usable privacy and security, and so my friends like to give me examples of their frustrations with computing systems, especially frustrations related to unusable privacy and security.

So passwords are something that I hear a lot about. A lot of people are frustrated with passwords, and it's bad enough when you have to have one really good password that you can remember but nobody else is going to be able to guess. But what do you do when you have accounts on a hundred different systems and you're supposed to have a unique password for each of these systems? It's tough.

At Carnegie Mellon, they used to make it actually pretty easy for us to remember our passwords. The password requirement up through 2009 was just that you had to have a password with at least one character. Pretty easy. But then they changed things, and at the end of 2009, they announced that we were going to have a new policy, and this new policy required passwords that were at least eight characters long, with an uppercase letter, lowercase letter, a digit, a symbol, you couldn't use the same character more than three times, and it wasn't allowed to be in a dictionary.

Now, when they implemented this new policy, a lot of people, my colleagues and friends, came up to me and they said, "Wow, now that's really unusable. Why are they doing this to us, and why didn't you stop them?"

And I said, "Well, you know what? They didn't ask me."

But I got curious, and I decided to go talk to the people in charge of our computer systems and find out what led them to introduce this new policy, and they said that the university had joined a consortium of universities, and one of the requirements of membership was that we had to have stronger passwords that complied with some new requirements, and these requirements were that our passwords had to have a lot of entropy. Now entropy is a complicated term, but basically it measures the strength of passwords. But the thing is, there isn't actually a standard measure of entropy. Now, the National Institute of Standards and Technology has a set of guidelines which have some rules of thumb for measuring entropy, but they don't have anything too specific, and the reason they only have rules of thumb is it turns out they don't actually have any good data on passwords. In fact, their report states, "Unfortunately, we do not have much data on the passwords users choose under particular rules. NIST would like to obtain more data on the passwords users actually choose, but system administrators are understandably reluctant to reveal password data to others."

So this is a problem, but our research group looked at it as an opportunity. We said, "Well, there's a need for good password data. Maybe we can collect some good password data and actually advance the state of the art here.

So the first thing we did is, we got a bag of candy bars and we walked around campus and talked to students, faculty and staff, and asked them for information about their passwords. Now we didn't say, "Give us your password." No, we just asked them about their password. How long is it? Does it have a digit? Does it have a symbol? And were you annoyed at having to create a new one last week? So we got results from 470 students, faculty and staff, and indeed we confirmed that the new policy was very annoying, but we also found that people said they felt more secure with these new passwords. We found that most people knew they were not supposed to write their password down, and only 13 percent of them did, but disturbingly, 80 percent of people said they were reusing their password. Now, this is actually more dangerous than writing your password down, because it makes you much more susceptible to attackers. So if you have to, write your passwords down, but don't reuse them. We also found some interesting things about the symbols people use in passwords. So CMU allows 32 possible symbols, but as you can see, there's only a small number that most people are using, so we're not actually getting very much strength from the symbols in our passwords.

So this was a really interesting study, and now we had data from 470 people, but in the scheme of things, that's really not very much password data, and so we looked around to see where could we find additional password data? So it turns out there are a lot of people going around stealing passwords, and they often go and post these passwords on the Internet. So we were able to get access to some of these stolen password sets. This is still not really ideal for research, though, because it's not entirely clear where all of these passwords came from, or exactly what policies were in effect when people created these passwords. So we wanted to find some better source of data. So we decided that one thing we could do is we could do a study and have people actually create passwords for our study. So we used a service called Amazon Mechanical Turk, and this is a service where you can post a small job online that takes a minute, a few minutes, an hour, and pay people, a penny, ten cents, a few dollars, to do a task for you, and then you pay them through Amazon.com. So we paid people about 50 cents to create a password following our rules and answering a survey, and then we paid them again to come back two days later and log in using their password and answering another survey. So we did this, and we collected 5,000 passwords, and we gave people a bunch of different policies to create passwords with. So some people had a pretty easy policy, we call it Basic8, and here the only rule was that your password had to have at least eight characters. Then some people had a much harder policy, and this was very similar to the CMU policy, that it had to have eight characters including uppercase, lowercase, digit, symbol, and pass a dictionary check. And one of the other policies we tried, and there were a whole bunch more, but one of the ones we tried was called Basic16, and the only requirement here was that your password had to have at least 16 characters.

All right, so now we had 5,000 passwords, and so we had much more detailed information. Again we see that there's only a small number of symbols that people are actually using in their passwords. We also wanted to get an idea of how strong the passwords were that people were creating, but as you may recall, there isn't a good measure of password strength. So what we decided to do was to see how long it would take to crack these passwords using the best cracking tools that the bad guys are using, or that we could find information about in the research literature.

So to give you an idea of how bad guys go about cracking passwords, they will steal a password file that will have all of the passwords in kind of a scrambled form, called a hash, and so what they'll do is they'll make a guess as to what a password is, run it through a hashing function, and see whether it matches the passwords they have on their stolen password list. So a dumb attacker will try every password in order. They'll start with AAAAA and move on to AAAAB, and this is going to take a really long time before they get any passwords that people are really likely to actually have. A smart attacker, on the other hand, does something much more clever. They look at the passwords that are known to be popular from these stolen password sets, and they guess those first. So they're going to start by guessing "password," and then they'll guess "I love you," and "monkey," and "12345678," because these are the passwords that are most likely for people to have. In fact, some of you probably have these passwords. So what we found by running all of these 5,000 passwords we collected through these tests to see how strong they were, we found that the long passwords were actually pretty strong, and the complex passwords were pretty strong too. However, when we looked at the survey data, we saw that people were really frustrated by the very complex passwords, and the long passwords were a lot more usable, and in some cases, they were actually even stronger than the complex passwords. So this suggests that, instead of telling people that they need to put all these symbols and numbers and crazy things into their passwords, we might be better off just telling people to have long passwords. Now here's the problem, though: Some people had long passwords that actually weren't very strong. You can make long passwords that are still the sort of thing that an attacker could easily guess. So we need to do more than just say long passwords. There has to be some additional requirements, and some of our ongoing research is looking at what additional requirements we should add to make for stronger passwords that also are going to be easy for people to remember and type.

Another approach to getting people to have stronger passwords is to use a password meter. Here are some examples. You may have seen these on the Internet when you were creating passwords. We decided to do a study to find out whether these password meters actually work. Do they actually help people have stronger passwords, and if so, which ones are better? So we tested password meters that were different sizes, shapes, colors, different words next to them, and we even tested one that was a dancing bunny. As you type a better password, the bunny dances faster and faster. So this was pretty fun.

What we found was that password meters do work. (Laughter) Most of the password meters were actually effective, and the dancing bunny was very effective too, but the password meters that were the most effective were the ones that made you work harder before they gave you that thumbs up and said you were doing a good job, and in fact we found that most of the password meters on the Internet today are too soft. They tell you you're doing a good job too early, and if they would just wait a little bit before giving you that positive feedback, you probably would have better passwords.

Now another approach to better passwords, perhaps, is to use pass phrases instead of passwords. So this was an xkcd cartoon from a couple of years ago, and the cartoonist suggests that we should all use pass phrases, and if you look at the second row of this cartoon, you can see the cartoonist is suggesting that the pass phrase "correct horse battery staple" would be a very strong pass phrase and something really easy to remember. He says, in fact, you've already remembered it. And so we decided to do a research study to find out whether this was true or not. In fact, everybody who I talk to, who I mention I'm doing password research, they point out this cartoon. "Oh, have you seen it? That xkcd. Correct horse battery staple." So we did the research study to see what would actually happen.

So in our study, we used Mechanical Turk again, and we had the computer pick the random words in the pass phrase. Now the reason we did this is that humans are not very good at picking random words. If we asked a human to do it, they would pick things that were not very random. So we tried a few different conditions. In one condition, the computer picked from a dictionary of the very common words in the English language, and so you'd get pass phrases like "try there three come." And we looked at that, and we said, "Well, that doesn't really seem very memorable." So then we tried picking words that came from specific parts of speech, so how about noun-verb-adjective-noun. That comes up with something that's sort of sentence-like. So you can get a pass phrase like "plan builds sure power" or "end determines red drug." And these seemed a little bit more memorable, and maybe people would like those a little bit better. We wanted to compare them with passwords, and so we had the computer pick random passwords, and these were nice and short, but as you can see, they don't really look very memorable. And then we decided to try something called a pronounceable password. So here the computer picks random syllables and puts them together so you have something sort of pronounceable, like "tufritvi" and "vadasabi." That one kind of rolls off your tongue. So these were random passwords that were generated by our computer.

So what we found in this study was that, surprisingly, pass phrases were not actually all that good. People were not really better at remembering the pass phrases than these random passwords, and because the pass phrases are longer, they took longer to type and people made more errors while typing them in. So it's not really a clear win for pass phrases. Sorry, all of you xkcd fans. On the other hand, we did find that pronounceable passwords worked surprisingly well, and so we actually are doing some more research to see if we can make that approach work even better. So one of the problems with some of the studies that we've done is that because they're all done using Mechanical Turk, these are not people's real passwords. They're the passwords that they created or the computer created for them for our study. And we wanted to know whether people would actually behave the same way with their real passwords.

So we talked to the information security office at Carnegie Mellon and asked them if we could have everybody's real passwords. Not surprisingly, they were a little bit reluctant to share them with us, but we were actually able to work out a system with them where they put all of the real passwords for 25,000 CMU students, faculty and staff, into a locked computer in a locked room, not connected to the Internet, and they ran code on it that we wrote to analyze these passwords. They audited our code. They ran the code. And so we never actually saw anybody's password.

We got some interesting results, and those of you Tepper students in the back will be very interested in this. So we found that the passwords created by people affiliated with the school of computer science were actually 1.8 times stronger than those affiliated with the business school. We have lots of other really interesting demographic information as well. The other interesting thing that we found is that when we compared the Carnegie Mellon passwords to the Mechanical Turk-generated passwords, there was actually a lot of similarities, and so this helped validate our research method and show that actually, collecting passwords using these Mechanical Turk studies is actually a valid way to study passwords. So that was good news.

Okay, I want to close by talking about some insights I gained while on sabbatical last year in the Carnegie Mellon art school. One of the things that I did is I made a number of quilts, and I made this quilt here. It's called "Security Blanket." (Laughter) And this quilt has the 1,000 most frequent passwords stolen from the RockYou website. And the size of the passwords is proportional to how frequently they appeared in the stolen dataset. And what I did is I created this word cloud, and I went through all 1,000 words, and I categorized them into loose thematic categories. And it was, in some cases, it was kind of difficult to figure out what category they should be in, and then I color-coded them.

So here are some examples of the difficulty. So "justin." Is that the name of the user, their boyfriend, their son? Maybe they're a Justin Bieber fan. Or "princess." Is that a nickname? Are they Disney princess fans? Or maybe that's the name of their cat. "Iloveyou" appears many times in many different languages. There's a lot of love in these passwords. If you look carefully, you'll see there's also some profanity, but it was really interesting to me to see that there's a lot more love than hate in these passwords. And there are animals, a lot of animals, and "monkey" is the most common animal and the 14th most popular password overall. And this was really curious to me, and I wondered, "Why are monkeys so popular?" And so in our last password study, any time we detected somebody creating a password with the word "monkey" in it, we asked them why they had a monkey in their password. And what we found out—we found 17 people so far, I think, who have the word "monkey"—We found out about a third of them said they have a pet named "monkey" or a friend whose nickname is "monkey," and about a third of them said that they just like monkeys and monkeys are really cute. And that guy is really cute.

So it seems that at the end of the day, when we make passwords, we either make something that's really easy to type, a common pattern, or things that remind us of the word password or the account that we've created the password for, or whatever. Or we think about things that make us happy, and we create our password based on things that make us happy. And while this makes typing and remembering your password more fun, it also makes it a lot easier to guess your password. So I know a lot of these TED Talks are inspirational and they make you think about nice, happy things, but when you're creating your password, try to think about something else.

Thank you.

(Applause)